Multimedia systems integrate multiple sources of digitally represented audio and video for simultaneous, synchronized presentation over an integrated set of output devices. For example, audio information might be stored on magnetic tape or a compact disc (CD), or introduced directly into the presentation from a microphone coupled to a digital sampler; video information can originate with a videodisc, CD (possibly, although not necessarily, the same CD containing audio information), or an active digital video camera. A multimedia presentation can also include graphical components, text, and still images, all stored digitally and introduced into the presentation at appropriate times. The presentation is conveyed to a viewer over output devices capable of converting the various digital representations into their native form. Multimedia output systems can range, for example, from no more than a computer-driven video monitor and associated sound hardware to integrated high-end stereo and digital video projection systems.
Managing the retrieval, integration, synchronization and overall flow of multimedia information, which can involve prodigious amounts of digital data even for relatively short presentation sequences, represents a substantial task. Ordinarily, computational responsibility for the various routing and control aspects of multimedia presentations is shared between a high-level multimedia application program, with which the user interacts to compose, edit, and view the presentation, and a multimedia operating system that performs basic, low-level operations such as data retrieval and output-device designation. This architecture simplifies the creation of multimedia applications by relieving programmers of the need to repeatedly specify routine data-handling operations at the computer system's most basic level; the multimedia operating system acts as a bridge between high-level, user-oriented application tasks and low-level input/output (I/O) and processor commands.
Among the tasks performed by the multimedia operating system is designation and control of output devices. Ordinarily, even digitally operated output devices do not accept streams of raw digital data directly, but instead operate through associated "device drivers." These modules, which ordinarily are specific to the devices they control, manage the flow of incoming data (perhaps converting it into a more appropriate form of electrical signal) and issue the control signals that actually operate the devices.
Recognizing the growing importance of multimedia applications, designers of basic computer operating sysems have upgraded their systems to include specific facilities for multimedia support. These systems effectively have "built-in" multimedia architectures that programmers can invoke as appropriate. Representative of such systems is the IBM Multimedia Presentation Manager/2.TM. (MMPM/2.TM.), supplied by International Business Machines Corporation (IBM) as an extension to its OS/2.RTM. operating system.
Important capabilities of MMPM/2 include synchronizing the sources of multimedia data and ensuring that constant streams of data are provided to active device drivers, thereby avoiding gaps in presentation. Because of differences in data-handling rates by various system components, the latter task can be complex. For example, audio data might be retrieved from mass storage in discrete blocks, each requiring a minimum disk access and data-retrieval time, and used to drive an output amplifier whose processing rate depends on the manner in which audio is digitally represented (e.g., the rate at which it was originally sampled). Were no provision made to reconcile the variety of data-handling rates, each mass-storage access sequence would produce a gap in presentation, and the device driver (which frequently outputs data more slowly than it can be retrieved from mass storage) might quickly become overloaded with data.
MMPM/2 addresses this problem with a buffering scheme that stores in volatile memory blocks of data retrieved from mass storage. The device driver obtains data from the buffer instead of from mass storage, and control software ensures that the buffer contains at least some data throughout presentation, thereby avoiding output interruptions.
Unfortunately, the data-handling problem increases substantially in complexity when additional features are introduced into the system, particularly during presentation. Suppose, returning to the above example, that a user wishes to introduce some type of audio filter or effect (e.g., an echo filter) even as audio output is being retrieved and provided to the output amplifier. A gap in presentation would necessarily result from the computational operations that divert the audio data stream from the output device to the filter, which has a characteristic throughput delay time that augments the gap. Similarly, were the user to remove the filter or effect in real-time, a new gap would be introduced as the stream is re-diverted directly to the output device. Straightforward buffering schemes, such as that of MMPM/b 2, cannot reliably eliminate this limitation.